Dynamically characterizing computer system performance by varying multiple input variables simultaneously

ABSTRACT

One embodiment of the present invention provides a system that dynamically characterizes computer system performance. The system operates by simultaneously varying multiple input variables in the computer system, and gathering performance results by measuring time-dependent responses in one or more response variables. In this way, responses to variations in the multiple input variables can be measured simultaneously. Next, the system analyzes the performance results to determine correlations between input variables and output variables.

RELATED APPLICATION

[0001] The subject matter of this application is related to the subjectmatter in a co-pending non-provisional application by Kenny C. Gross andDavid M. Fishman, entitled, “Method and Apparatus To FacilitateMeasurement of Quality-Of-Service Performance of a Network Server,” Ser.No. 09/887,944, filed Jun. 22, 2001 (Attorney Docket No. SUN-P5595-RSH).

BACKGROUND

[0002] 1. Field of the Invention

[0003] The present invention relates to the process of measuring theperformance of computer systems. More specifically, the presentinvention relates to a method and an apparatus for dynamicallycharacterizing computer system performance by varying multiple inputvariables simultaneously.

[0004] 2. Related Art

[0005] Current methods for qualification testing of enterprise computingsystems involve putting a maximum expected load on one or more inputvariables, and seeing if the system crashes. While this type ofqualification testing is necessary, dynamical system characterizationcan provide a far greater wealth of information that is useful fordesigning robust systems that deliver optimal Quality-of-Service (QOS)performance over a large range of system parameters.

[0006] Dynamical system characterization involves introducingperturbations in one or more “input” variables, and measuring thetime-dependent responses in one or more “response” variables. Typicalinput variables of interest can include those associated with load onthe CPU, load on the memory, or I/O traffic to disk. Typical responsevariables of interest can include physical variables such as localtemperatures, voltages, or currents; or a large range of what are calledQuality-of-Service (QOS) variables. QOS variables can also include avariety of “wait times” that users experience when interacting withlarge Web centers. For example, QOS variables may include: the wait timeafter clicking on a shopping cart; the time to return a database query;or the time to process an FTP request to download a file.

[0007] Dynamical system characterization asks, “For a perturbation of x% in input variable A, what is the variation in response variable B?” Wequantify this relationship with a “dynamic coupling coefficient” betweenA and B. This coupling coefficient, C, may be a function of load, or,more generally, may be a multivariate function of multiple inputvariables. There may be a linear relationship between input A andresponse B; or the relationship may be highly nonlinear.

[0008] One can make an analogy with structural mechanics for airplanes.Static load testing may show that structural elements are strong enoughto withstand all expected loads for a plane. However, resonances mayexist wherein small vibrations are nonlinearly amplified, leading tocatastrophic failures. Early airplane designers learned that it iscrucially important to do dynamic response testing to learn if suchstructural resonances exist.

[0009] For computing systems, numerous phenomena have been identifiedthat can lead to nonlinear coupling between small fluctuations in inputvariables and response variables. For example, if memory usage is nearthe limit of available memory, applications will swap to disk, a processthat is much slower. For this reason, when stochastic load variationsoccur with lots of available memory remaining, coupling with QOSresponse variables is linear, has small coupling coefficients, and hasextremely small phase-shifts (lag times) between input and responsevariables. However, when the same stochastic load variations occur nearthe limit of available memory, there is a strongly nonlinear couplingbetween input and response variables, and with a measurable phase-shift.

[0010] Similar nonlinear coupling between input variables and responsevariables has been observed in online transaction processing (OLTP)systems with a phenomenon called latch contention. Moreover, innetworked systems that are bandwidth constrained, packet collisions canlead to non-linearities in QOS latencies. Hence, there are manyphenomena within computer systems that can lead to nonlinear dynamicalinteraction effects between input variables and a multitude of physicaland QOS response variables.

[0011] Hence, what is needed is a method and an apparatus fordynamically characterizing computer system performance.

[0012] Although existing tools can measure and log these QOS variablesas a function of time, there are no tools that permit the diagnosis ofinteraction effects between and among the various QOS variables. Asystem analyst may be interested in the answers to questions such as:(1) If I have 10,000 email users this month, and I add 2,000 new emailaccounts next month, how will the increased email traffic impact mydatabase users' wait times? (2) What impact do 8,000 animated browserbanners have on download latency times for my 3,000 active file transferprotocol (FTP) users—and vice versa? (3) What impact do 4,000 activeportable document format (PDF) file viewers have on transactionprocessing system (TPS) throughput? In general, what impact does theinstantaneous demand for performance variable X produce in performancevariable Y?

[0013] Attempting to measure the impact that one variable has on anothervariable is difficult, at best. One way of attempting to measure thisinteraction is to create a step increase in the number of transactionsof one category and search for a change in the response, or wait, timein transactions of another category. The ability to find a change in thewait time in a second category of transaction requires the step increasein the first category of transaction to be relatively large. However,using a large step increase is undesirable because the large stepincrease may adversely affect the response time in the first variable atthe same time. Additionally, if the step increase is too large, theincrease can cause the system to fail, or “crash.”

[0014] Hence, what is needed is a method and an apparatus to quantifythe interaction of QOS variables.

[0015] Another problem is that the process of dynamically characterizingcomputer system performance can be extremely time-consuming. Forexample, a test involving a single input variable can require up toseveral hours. Consequently, a test involving multiple input variablescan potentially take many days, which is an unacceptable amount of timein most testing contexts.

[0016] What is needed is a method and an apparatus for dynamicallycharacterizing computer system performance through tests involvingmultiple input variables without requiring an undue amount of time.

SUMMARY

[0017] One embodiment of the present invention provides a system thatdynamically characterizes computer system performance. The systemoperates by simultaneously varying multiple input variables in thecomputer system, and gathering performance results by measuringtime-dependent responses in one or more response variables. In this way,responses to variations in the multiple input variables can be measuredsimultaneously. Next, the system analyzes the performance results todetermine correlations between input variables and output variables.

[0018] In a variation on this embodiment, varying the multiple inputvariables involves using the concentric-hypersphere perturbationtechnique to generate very nearly ideal sinusoidal impulsionalperturbations in the multiple input variables simultaneously.

[0019] In a variation on this embodiment, varying the multiple inputvariables involves varying a single input variable using a compositeperiodic disturbance comprising multiple distinct frequencies.

[0020] In a variation on this embodiment, varying the multiple inputvariables involves using multiple amplitudes for varying a single inputvariable.

[0021] In a variation on this embodiment, analyzing the performanceresults involves performing a normalized cross power spectral density(NCPSD) analysis on the performance results.

[0022] In a variation on this embodiment, simultaneously varying themultiple input variables involves generating a pattern of synthetictransactions, and then sending the pattern of synthetic transactions tothe computer system to be processed. (Note that “synthetic transactions”are transactions that are generated for the performance measurementpurposes, and are not part of the computer system's normal workload.)

[0023] In a further variation, the pattern of synthetic transactions issent to the computer system while the computer system is processing anormal workload, so that the pattern of synthetic transactions is addedto the normal workload.

[0024] In a further variation, the pattern of synthetic transactionsincludes a varying pattern of synthetic transactions.

[0025] In a further variation, the pattern of transactions additionallyincludes a fixed pattern of synthetic transactions.

BRIEF DESCRIPTION OF THE FIGURES

[0026]FIG. 1 illustrates a number of clients coupled to a number ofservers in accordance with an embodiment of the present invention.

[0027]FIG. 2 illustrates a synthetic transaction generator in accordancewith an embodiment of the present invention.

[0028]FIG. 3 presents a flowchart illustrating the process of generatingsynthetic transactions and measuring response times in accordance withan embodiment of the present invention.

[0029]FIG. 4 presents a flowchart illustrating the process ofquantifying the effects of transactions of one category on transactionsof other categories in accordance with an embodiment of the presentinvention.

[0030]FIG. 5 is a flowchart illustrating the process of dynamicallycharacterizing system performance by varying multiple input variablessimultaneously in accordance with an embodiment of the presentinvention.

[0031]FIG. 6 illustrates the concentric-hypersphere perturbationtechnique in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION

[0032] The following description is presented to enable any personskilled in the art to make and use the invention, and is provided in thecontext of a particular application and its requirements. Variousmodifications to the disclosed embodiments will be readily apparent tothose skilled in the art, and the general principles defined herein maybe applied to other embodiments and applications without departing fromthe spirit and scope of the present invention. Thus, the presentinvention is not limited to the embodiments shown, but is to be accordedthe widest scope consistent with the principles and features disclosedherein.

[0033] The data structures and code described in this detaileddescription are typically stored on a computer readable storage medium,which may be any device or medium that can store code and/or data foruse by a computer system. This includes, but is not limited to, magneticand optical storage devices such as disk drives, magnetic tape, CDs(compact discs) and DVDs (digital versatile discs or digital videodiscs), and computer instruction signals embodied in a transmissionmedium (with or without a carrier wave upon which the signals aremodulated). For example, the transmission medium may include acommunications network, such as the Internet.

[0034] Clients and Servers

[0035]FIG. 1 illustrates a number of clients 106-109 coupled to a numberof servers 103-105 through a network 102 in accordance with anembodiment of the present invention.

[0036] Network 102 can generally include any type of wire or wirelesscommunication channel capable of coupling together computing nodes. Thisincludes, but is not limited to, a local area network, a wide areanetwork, or a combination of networks. In one embodiment of the presentinvention, network 102 includes the Internet.

[0037] Clients 106-109 can generally include any node on network 102including computational capability and including a mechanism forcommunicating across the network. Clients 106-109 can be positioned atsites that are geographically separated from servers 103-105 and fromeach other. Servers 103-105 can generally include any nodes on network102 including a mechanism for servicing requests from clients 106-109for computational and/or data storage resources.

[0038] Clients 106-109 and servers 103-105 reside in computer systems,which can generally include any type of computer system, including, butnot limited to, a computer system based on a microprocessor, a mainframecomputer, a digital signal processor, a portable computing device, apersonal organizer, a device controller, and a computational enginewithin an appliance.

[0039] Clients 108 and 109 include synthetic transaction generators 110and 111, respectively. During operation, synthetic transactiongenerators 110 and 111 generate synthetic transactions for servers103-105. These synthetic transactions are sent to servers 103-105 acrossnetwork 102 and are processed up to the point where the transactionwould be committed. The transaction is then cancelled, saving only theresponse time for the transaction to be committed.

[0040] Synthetic transaction generators 110 and 111 generate synthetictransactions for the categories of services that are available onservers 103-105. These synthetic transactions can include, but are notlimited to, file transfers, downloads, database queries, emailtransactions, browser advertising banners, office applications, PDF fileviewers, calendar updates, code executions, and file compressions.

[0041] The synthetic transaction generators place small, typicallysinusoidal, variations in the number of transactions for one category ofservice while holding the number of transactions in the other categoriesof service constant. The response times for the synthetic transactionsare stored in a table, which is subsequently analyzed using mathematicaltechniques to determine the effects on response times in the othercategories of service caused by the variations in the number oftransactions in the category of service being varied. The number ofsynthetic transactions can be low—an order of magnitude or morelower—compared to the number of real transactions being handled byservers 103-105 in order to prevent the synthetic transactions fromsignificantly affecting server performance. Moreover, the period of thesinusoid that is varying the synthetic transactions can be very long,possibly measured in hours.

[0042] Furthermore, synthetic transaction generators 110 and 111 canpossibly generate transaction workloads that vary more than one inputvariable at a time in order to measure the performance of servers103-105. This process is described in more detail below with referenceto FIGS. 5-6.

[0043] Although the present invention is described in the context of acomputer system that includes multiple servers, the present inventioncan generally be applied to measuring the performance of any type ofcomputer system. For example, the present invention can be applied tomeasure the performance of a single server, or to measure theperformance of a standalone computer system that is not coupled to anetwork.

[0044] Synthetic Transaction Generator

[0045]FIG. 2 illustrates synthetic transaction generator 110 inaccordance with an embodiment of the present invention. Synthetictransaction generators 110 and 111 from FIG. 1 are similar, so onlysynthetic transaction generator 110 will be discussed herein. Synthetictransaction generator 110 contains a number of components, includingsinusoidal transaction generator 202, fixed transaction generator 204,transaction processor 206, response time measurer 208, and dependencycalculator 210.

[0046] Sinusoidal transaction generator 202 generates synthetictransactions to be applied to servers 103-105 across network 102. Thenumber of synthetic transactions varies sinusoidally over time. Thesetransactions are applied to one category of service as discussed abovein conjunction with FIG. 1.

[0047] In one embodiment of the present invention, each category ofservice being varied by sinusoidal transaction generator 202 is cycled,individually over time, so that each category of service can beanalyzed.

[0048] In another embodiment of the present invention, multiplecategories of service are varied simultaneously as is described belowwith reference to FIGS. 5 and 6.

[0049] Fixed transaction generator 204 provides a fixed number oftransactions over time for the other categories of services beingprovided by servers 103-105. These fixed rate transactions provideresponse time measurements for the categories of service other than thecategory being controlled. The system uses these response times tocalculate dependencies between the category of service being controlledand the other categories of service as described below in conjunctionwith FIG. 4.

[0050] Transaction processor 206 aggregates the synthetic transactionsfrom sinusoidal transaction generator 202 and from fixed transactiongenerator 204 and sends them to servers 103-105 across network 102.Transaction processor 206 processes each transaction up to the pointwhere servers 103-105 are ready to commit the transaction. At thispoint, the transaction is cancelled.

[0051] Response time measurer 208 operates in conjunction withtransaction processor 206 to determine the response time—or waittime—associated with the synthetic transaction. Response time measurer208 saves all of the response times in a table for processing bydependency calculator 210.

[0052] Dependency calculator 210 calculates the dependencies between theresponse time for each service category as is described below inconjunction with FIG. 4.

[0053] Measuring Transaction Response Times for a Single Category

[0054]FIG. 3 is a flowchart illustrating the process of generatingsynthetic transactions and measuring response times in accordance withan embodiment of the present invention. The system starts when synthetictransaction generator 110 selects a service category to be controlled(step 302). Sinusoidal transaction generator 202 then generates asinusoidally varying pattern of synthetic transactions for this servicecategory (step 304). At the same time, fixed transaction generator 204generates a fixed pattern of synthetic transactions for all otherservice categories supplied by servers 103-105 (step 306).

[0055] Next, transaction processor 206 applies these synthetictransactions to servers 103-105 across network 102 (step 308). Asdescribed above in conjunction with FIG. 2, transaction processor 206processes each transaction up to the point where servers 103-105 areready to commit the transaction. At this point, the transaction iscancelled. Note that the period of the sinusoid used to vary the numberof synthetic transactions can be very long, possibly measured in hours.Response time measurer 208 measures the response times for thesetransactions and stores the response times as wait times in a table(step 310). This table is indexed by time of transaction and containsthe wait times for the various categories of service. In one embodimentof the present invention, a different table is created for each servicetype.

[0056] Finally, synthetic transaction generator 110 determines if allservice categories have been varied by sinusoidal transaction generator202 (step 312). If not, the process returns to 302 to select anotherservice category. Otherwise, the process is complete.

[0057] In one embodiment of the present invention, synthetic transactiongenerator 110 generates a synthetic workload that is sent to servers103-105 while servers 103-105 are processing a normal workload, so thatthe pattern of synthetic transactions is added to the normal workload

[0058] Determining Dependencies for a Single Category at a Time

[0059]FIG. 4 is a flowchart illustrating the process of quantifying theeffects of transactions of one category on transactions of othercategories in accordance with an embodiment of the present invention.The system starts when dependency calculator 210 within synthetictransaction generator 110 receives tables of wait times from responsetime measurer 208 for all service types (step 402).

[0060] Note that the transactions generated by the fixed transactiongenerator may occur at intervals that are not quite fixed intervalsbecause it is difficult to get a particular action to occur at a precisetime. To compensate for this difficulty, Cubic Spline Interpolation maybe used. Cubic Spline Interpolation is well-known in the art, and isavailable in MatLab™ (MatLab™ is a product of The MathWorks, Natick,Mass.). Next, dependency calculator 210 sets counter I to 1 (step 404).

[0061] Also note that this description uses counters I and J to controlthe process. The counter I indexes the service category being controlledby sinusoidal transaction generator 202, while the counter J indexes allof the other categories of service offered by servers 103-105. However,note that mechanisms other than counters can be used to control thisprocess.

[0062] Next, dependency calculator 210 selects the table associated withservice category I (step 406). After selecting the table, dependencycalculator 210 sets counter J to 1 to select a column from the table toanalyze for dependency on service category I (step 408). Dependencycalculator 210 then performs a bivariate normalized cross power spectraldensity computation (NCPSD) on the pair of wait times (step 410).Details of the bivariate NCPSD computation are well known in the art andcan be found in Spectral Analysis and Its Applications by Jenkins, G. M.and Watts, D. G. (1968, Holden-Day, San Francisco, Calif.).

[0063] After calculating the NCPSD, dependency calculator 210 performs aKolmagrov-Smirnov test for whiteness to determine if a dependency existsbetween transactions of one category of service and transactions ofother categories of service (step 412). The Kolmagrov-Smirnov test is astandard frequency domain statistical test that is well known in the artand can be found in An Introduction to Stochastic Processes, 2^(nd) Ed.(1966, Cambridge University Press). Next, dependency calculator 210determines if a dependency was detected by the Kolmagrov-Smirnov test(step 414).

[0064] If a dependency was detected, dependency calculator 210 storesthe values of both counters I and J, and the value returned from theNCPSD (step 416). Next, dependency calculator 210 computes and storesthe transfer function coefficients at the excitation frequency orfrequencies used by sinusoidal transaction generator 202 to drive theservice category (step 418). Note that these transfer functioncoefficients are the final output of the system and can be used by asystem analyst to determine the effects that one service has on othercategories of service supplied by servers 103-105.

[0065] After computing the transfer functions at step 418, or if nodependency was detected at step 414, dependency calculator 210increments counter J (step 420). Next, dependency calculator 210determines if counter J has been incremented past the last availableservice (step 422). If not, the process returns to step 410 to calculatethe next NCPSD.

[0066] If counter J has been incremented past the last availableservice, dependency calculator 210 increments counter I and resets thecounter J to a value of one (step 424). After incrementing counter I,dependency calculator 210 determines if counter I has been incrementedpast the last service being tested for creating a dependency (step 426).If so, the process is ended, otherwise the process returns to 406 toselect the next table.

[0067] Dynamically Characterizing Multiple Input VariablesSimultaneously

[0068]FIG. 5 is a flow chart illustrating the process of dynamicallycharacterizing system performance by varying multiple input variablessimultaneously in accordance with an embodiment of the presentinvention. This process works similarly to the process for varying asingle input variable discussed above, except that instead of varyingonly a single input variable, multiple input variables are varied at thesame time.

[0069] During operation, the system generates a load that variesmultiple input variables simultaneously (step 502). The systemsimultaneously measures responses in multiple output variables (step504). Next, the system analyzes the responses to determine correlationsbetween input variables and output variables (step 506).

[0070] While generating the load in step 502, one embodiment of thepresent invention uses a multi-frequency sinusoidal excitation approachto superimpose a periodic perturbation with very small amplitude on X.The spectral decomposition of resulting responses in other performancevariables Y(i), i=1, 2, 3, . . . , N, determines the couplingcoefficients among all elements of the system, thereby characterizingthe dynamical system behavior.

[0071] In order to diagnose the dependability interactions among systemparameters, the periodic variation introduced in the system may besinusoidal with respect to time. One embodiment of the present inventionuses the concentric-hypersphere perturbation technique to generatenearly ideal sinusoidal impulsional perturbations in multiple inputvariables simultaneously. This technique employs an advancedmathematical technique called Gaussian Quadrature, which was originallydeveloped for discrete-ordinates approximations to neutron diffusionequations for high-energy physics. See “Concentric-Sphere Design forSpacing of Tag-Gas Isotopic Ratios,” K. C. Gross, Nuclear TechnologyJournal, Vol. 45 (September 1979).

[0072] Once multi-frequency sinusoidal perturbations have beensuperimposed on desired input parameters, a spectral decompositionanalysis during step 506 determines the correlation among systemvariables by calculating the signal amplification in a periodogram,which plots the power spectral density of the signal as a function ofperiod (with units of time). Note that for noisy process variables withlarge random components, conventional time-domain cross-correlationmethods fail.

[0073] The technique introduced above can accurately assesscross-correlation among multiple dynamic system parameters, even withextremely small signal-to-noise ratios. In addition to cross-correlationamong response variables, the technique also provides: dynamicalcoupling coefficients between input variables and any number of responsevariables; and phase shifts (which can be expressed differently to givelag times) between “cause” and “effect” variables throughout the dynamicsystem.

[0074] Laboratory experiments have shown that systems with chaoticdynamics and high noise levels are not amenable to univariate spectraldecomposition calculations via conventional Fourier analysis because thesignal-to-noise ratio is too small to discern the sinusoidalperturbation in the response variables. To overcome this limitation ofconventional Fourier analysis methods, one embodiment of the presentinvention employs the Normalized Cross Power Spectral Density (NCPSD)technique. See “Investigation of Nonrecoil Fission-Product ReleasePhenomena Using Multifrequency Source-Perturbation Experiments inEBR-II,” K. C. Gross and L. K. Polley, Annals of Nuclear Energy, Vol. 18(3), pp. 419-441 (1986). This bivariate diagnostic technique is highlysensitive even to weakly coupled parameters with very poorsignal-to-noise ratios. It dramatically and selectively amplifies theinput sinusoid harmonics in response variables such that the period ofthe sinusoidal perturbation in the control variable is readily apparentwith the sharpest peak resolution and lowest noise and “side lobe”contamination.

[0075] Multi-frequency sinusoidal parameter stimulation, coupled withNCPSD analysis, can provide richly detailed diagnostics withoutencountering the Heisenberg Effect (an undesirable effect wherein theact of measuring the dynamics of a system can distort—or in the worstcase, crash—the system under surveillance). Concentric hyperspheres thattrace out sinusoids has significant advantages for simultaneouslydriving coupled harmonics into two or more system input variables,allowing for monitoring of multiple input and output performanceparameters that fully characterize the dynamical behavior,diagnosability, and margining robustness for systems. Additionally, thisprocess can be applied to comprehensively determine true availabilitybudgeting and QOS metrics from the end user's perspective.

[0076] Concentric Hypersphere Perturbation Technique

[0077]FIG. 6 illustrates the concentric-hypersphere perturbationtechnique in accordance with an embodiment of the present invention. Thefirst step in generating a load that varies multiple input variablessimultaneously is to compute a set of geometric cosines, μ_(i), with theequation, $\begin{matrix}{{\mu_{i} = \left\lbrack {{\sin^{2}\frac{\pi}{4\quad N_{I}}} + {\frac{\left( {i - 1} \right)}{\left( {N_{I} - 1} \right)}\left( {1 - {3\quad \sin^{2}\frac{\pi}{4\quad N_{I}}}} \right)}} \right\rbrack^{\frac{1}{2}}},{i = 1},2,\quad \ldots \quad,{N_{I}.}} & (1)\end{matrix}$

[0078] Equation (1) is derived from angular Quadrature relationshipsused in discrete-ordinates approximations to the neutron transportequation. Here, N_(I), the number of cosines in the positive μdirection, is the subscript of the commonly used S_(n)discrete-ordinates scheme. Use of equation (1) ensures that theresulting system will posses complete symmetry with respect to theorigin, and that the distances between each node and all of its closestneighbors are equalized as nearly as possible.

[0079] Using the set of μ_(i) generated by equation (1), the locationsof the nodes for the sphere are determined from,

[0080] (X^(n), Y^(n), Z^(n))=(Rμ_(i), Rμ_(j), Rμ_(k)),

[0081] i=1, 2, . . . N_(I),

[0082] j−1, 2, . . . , (N_(I)−i+1),

[0083] k=N_(I)+2−i−j,

[0084] where (X^(n), Y^(n), Z^(n)) are the coordinates of node n on thesurface of the sphere, and where R is the radius of the sphere. Notethat as indices i and j vary from 1 to N_(I) and from 1 to (N_(I)−i+1),respectively, n varies from 1 to M_(I), where M_(I), the total number ofnodes on the sphere, is equal to 4N_(I)(N_(I)+1). FIG. 6 depicts a fewnodes from the positive octant of the sphere.

[0085] The foregoing descriptions of embodiments of the presentinvention have been presented only for purposes of illustration anddescription. They are not intended to be exhaustive or to limit thepresent invention to the forms disclosed. Accordingly, manymodifications and variations will be apparent to practitioners skilledin the art. Additionally, the above disclosure is not intended to limitthe present invention. The scope of the present invention is defined bythe appended claims.

What is claimed is:
 1. A method for dynamically characterizing computersystem performance by varying multiple input variables simultaneously,comprising: simultaneously varying multiple input variables to thecomputer system; gathering performance results by measuringtime-dependent responses in one or more response variables; wherebyresponses to variations in the multiple input variables can be measuredsimultaneously; and analyzing the performance results to determinecorrelations between input variables and output variables.
 2. The methodof claim 1, wherein varying the multiple input variables to the computersystem involves using the concentric-hypersphere perturbation techniqueto generate very nearly ideal sinusoidal impulsional perturbations inthe multiple input variables simultaneously.
 3. The method of claim 1,wherein varying the multiple input variables to the computer systeminvolves varying a single input variable using a composite periodicdisturbance comprising multiple distinct frequencies.
 4. The method ofclaim 1, wherein varying the multiple input variables to the computersystem involves using multiple amplitudes for varying a single inputvariable.
 5. The method of claim 1, wherein analyzing the performanceresults involves performing a normalized cross power spectral density(NCPSD) analysis on the performance results.
 6. The method of claim 1,wherein simultaneously varying the multiple input variables involves:generating a pattern of synthetic transactions; and sending the patternof synthetic transactions to the computer system.
 7. The method of claim6, wherein the pattern of transactions is sent to the computer systemwhile the computer system is processing a normal workload, so that thepattern of synthetic transactions is added to the normal workload. 8.The method of claim 6, wherein the pattern of synthetic transactionsincludes a varying pattern of synthetic transactions.
 9. The method ofclaim 8, wherein the pattern of transactions additionally includes afixed pattern of synthetic transactions.
 10. A computer-readable storagemedium storing instructions that when executed by a computer cause thecomputer to perform a method for dynamically characterizing computersystem performance by varying multiple input variables simultaneously,the method comprising: simultaneously varying multiple input variablesto the computer system; gathering performance results by measuringtime-dependent responses in one or more response variables; wherebyresponses to variations in the multiple input variables can be measuredsimultaneously; and analyzing the performance results to determinecorrelations between input variables and output variables.
 11. Thecomputer-readable storage medium of claim 10, wherein varying themultiple input variables to the computer system involves using theconcentric-hypersphere perturbation technique to generate very nearlyideal sinusoidal impulsional perturbations in the multiple inputvariables simultaneously.
 12. The computer-readable storage medium ofclaim 10, wherein varying the multiple input variables to the computersystem involves varying a single input variable using a compositeperiodic disturbance comprising multiple distinct frequencies.
 13. Thecomputer-readable storage medium of claim 10, wherein varying themultiple input variables to the computer system involves using multipleamplitudes for varying a single input variable.
 14. Thecomputer-readable storage medium of claim 10, wherein analyzing theperformance results involves performing a normalized cross powerspectral density (NCPSD) analysis on the performance results.
 15. Thecomputer-readable storage medium of claim 10, wherein simultaneouslyvarying the multiple input variables involves: generating a pattern ofsynthetic transactions; and sending the pattern of synthetictransactions to the computer system.
 16. The computer-readable storagemedium of claim 15, wherein the pattern of transactions is sent to thecomputer system while the computer system is processing a normalworkload, so that the pattern of synthetic transactions is added to thenormal workload.
 17. The computer-readable storage medium of claim 15,wherein the pattern of synthetic transactions includes a varying patternof synthetic transactions.
 18. The computer-readable storage medium ofclaim 17, wherein the pattern of transactions additionally includes afixed pattern of synthetic transactions.
 19. An apparatus thatdynamically characterizes computer system performance by varyingmultiple input variables simultaneously, comprising: a workloadgenerator that is configured to simultaneously vary multiple inputvariables to the computer system; a measurement mechanism that isconfigured to gather performance results by measuring time-dependentresponses in one or more response variables; whereby responses tovariations in the multiple input variables can be measuredsimultaneously; and an analysis mechanism that is configured to analyzethe performance results to determine correlations between inputvariables and output variables.
 20. The apparatus of claim 19, whereinthe workload generator is configured to use the concentric-hypersphereperturbation technique to generate very nearly ideal sinusoidalimpulsional perturbations in the multiple input variablessimultaneously.
 21. The apparatus of claim 19, wherein the workloadgenerator is configured to vary a single input variable using acomposite periodic disturbance comprising multiple distinct frequencies.22. The apparatus of claim 19, wherein the workload generator isconfigured to use multiple amplitudes for varying a single inputvariable.
 23. The apparatus of claim 19, wherein the analysis mechanismis configured to perform a normalized cross power spectral density(NCPSD) analysis on the performance results.
 24. The apparatus of claim19, wherein the workload generator is configured to: generate a patternof synthetic transactions; and to send the pattern of synthetictransactions to the computer system.
 25. The apparatus of claim 24,wherein the pattern of transactions is sent to the computer system whilethe computer system is processing a normal workload, so that the patternof synthetic transactions is added to the normal workload.
 26. Theapparatus of claim 24, wherein the pattern of synthetic transactionsincludes a varying pattern of synthetic transactions.
 27. The apparatusof claim 26, wherein the pattern of transactions additionally includes afixed pattern of synthetic transactions.